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Abstract 



> 

C***~ , We investigate the potential of a high-energy 77 collider to detect an 

o 

On 



anomalous tqj coupling from observation of the reaction 77 — ► tq, tq, where 



\ q = c or u. We find that with 6-tagging and suitable kinematic cuts this pro- 

P-t. 

Qh! cess should be observable if the anomalous coupling re/A is no less than about 



0.05/TeV, where A is the scale of new physics associated with the anomalous 



interaction. This improves upon the bound possible from observation of top 

h : 

decays at the Tevatron. 



I. INTRODUCTION 

Since the discovery of the top quark at the Fermilab Tevatron by the CDF and DO 
collaborations there has been much speculation as to whether or not its interactions are 
in accordance with Standard Model (SM) predictions. Because its mass, around 175 GeV, 
is of the order of the Fermi scale, the top quark couples quite strongly to the electroweak 
symmetry-breaking sector. In the minimal SM the electroweak symmetry-breaking sector 
consists of a single complex fundamental Higgs scalar, but "triviality" [fj and "naturalness" 
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of the scalar sector suggest that in fact the Higgs sector, and therefore the top quark 
mass generation mechanism, may be more complicated. It is therefore plausible to assume 
that the Higgs sector of the SM is an effective theory, and that new physics phenomena may 
be manifested through effective interactions of the top quark |4]]. 

One interesting sub-set of effective top quark interactions mediates flavour changing 
neutral t decays, i.e., t — > cZ, eg, and cy. The SM predictions for the corresponding 
branching fractions are unobservably small H; thus any experimental evidence for such 
decays will be an unambiguous signal for physics beyond the Standard Model. Furthermore, 
it has been argued that these decay rates may be enhanced significantly in many extensions 
of the standard model, such as SUSY or other models with multiple Higgs doublets [||||, 
models with new dynamical interactions of the top quark j7|, and models where the top 
quark has a composite || or soliton structure [|]. 

Many aspects of such anomalous top-quark couplings have already been investigated in 
hadron and lepton colliders. These couplings give contributions to low-energy observables 
such as the partial width ratio Rf, = T(Z — > bb)/T(Z — > hadrons) measured at LEP-I [ITU 



or the branching fraction for b — > 37 [|TTJ. Other constraints from low-energy processes on 
anomalous couplings of top quark have also been considered in the literature [I5|. Fur- 
thermore, such couplings would also affect top-quark production and decay processes at 
hadron and e + e~ colliders |13,|l4|. In Ref. |I5| , the experimental constraints on an anoma- 



lous top-quark coupling tcZ and the experimental observability of the induced rare decay 
mode t — > Zc, at the Fermilab Tevatron and the CERN LHC, have been investigated in 
detail. The observability of an anomalous coupling leg has been studied for t — > eg decays 
at the Tevatron [jlJJ, for single top production in association with a charm quark |]I7| , |18| , 
and for direct top production, gq — > t (q = c or u) at the Tevatron and the LHC [ |T9|j . 

In this paper, we examine the possibility of searching for the anomalous top-quark cou- 
plings icy and iw-f at a high-energy 77 collider. Such a collider may be constructed by the 
compton scattering of laser light off the e + and e~ beams in an e + e~ linear collider [J2U|J . We 
will consider the anomalous effective Lagrangian which includes only the lowest dimension, 



CP-conserving operators which give rise to anomalous tq'j vertices, namely 

AC eff = j[K c ta^c + K u ia^u]F^ + h.c, (1) 

where F^ u is the electromagnetic field strength tensor, e is the electromagnetic coupling 
constant, A is the cutoff of the effective theory, which is generally taken to be the order of 
1 TeV, and the parameters re c and k u can be interpreted as the strengths of the anomalous 
interactions. Here we do not choose a particular scale, and consider only the ratio re/A. In 
principle, one could consider a more complex form factor with the tensor structure a^A + 
-B7 5 ), however, if we consider only polarization averaged cross-sections, we may set B = 
without any loss of generality in probing the strength of such couplings. 



The constraint from the inclusive branching ratio for the process b — > 37 [^1|] on the 
anomalous top-quark coupling icy gives re c /A < 0.16/TeV in the absence of an anomalous 
icg coupling [[SJ (the limit with a non-zero icg coupling is 0.28/TeV). The analysis can be 
extended to the iwy coupling by realizing that the anomalous contribution to b — > 57 will be 
suppressed by the CKM factor V us ~ 0.22 compared to the icy case, which translates into 
the limit k u /A < 0.72/TeV. The current direct experimental bound comes from CDF data 
on the top branching fractions to a jet plus photon 



BF(t -> 07) + BF(t -> wy) < 2.9%, (2) 
at 95% Confidence Level (CL), which translates into the limit 



re/A = ^Jk 2 c + re2/A < (0.73/TeV)/ \J BF(t -> bW). (3) 

This experimental limit therefore does not improve on the b — > 57 constraint in either case, 
although the bounds on re u are numerically about the same. 

By searching for the decay t — > qj in ti production |23|] one can potentially observe 



the anomalous couplings down to re/A = 0.12/TeV at the Tevatron Upgrade with 10 fb _1 
of integrated luminosity and to re/A = 0.01/TeV at the LHC with 100 fb _1 of integrated 
luminosity, assuming that the light quark jet can not be identified. We will find that the 



77 collider may be able to improve upon the potential Tevatron limit on these anomalous 
couplings by roughly a factor of 2.5. The relatively clean environment of a 77 collider 
presents an opportunity for differentiating between the icy and iwy couplings, assuming 
c-tagging by the identification of D mesons produced by hadronization, such as is done 
in various e + e~ experiments P^j] , is possible. This particular option does not seem to be 
feasible at hadron colliders. 



II. ANOMALOUS SINGLE TOP PRODUCTION AT A 77 COLLIDER 

The anomalous vertices under consideration here lead to the interaction 77 — > tq, where 
q = c or u. There are four diagrams which contribute, each with one anomalous vertex and 
one SM vertex: either the top quark or light quark can be exchanged in the t or u channel. 
For very high energies, in the limit that both fermion masses can be ignored, the total cross 
section approaches (7(77 — > tq) = 647TO! 2 (k/A) 2 . This result is finite despite the t and u- 
channel poles because the momentum coupling in Eq. [I] regulates the divergence when the 
fermion masses vanish. For k/A — 0.16/TeV, the maximal value allowed for the icy coupling 
in the absence of a iqg coupling, this gives about 120 fb (for the corresponding iuy coupling 
this is increased by roughly a factor of 20). In this paper we will use k/A = 0.16/TeV in 
our calculations of the signal unless noted otherwise. The cross section for single antitop 
production 77 — > iq is the same; henceforth in this paper all signal rates and discussions 
will include the sum of the single top and antitop signals. The cleanest signal occurs when 
the top quark decays semileptonically, which gives the signature bjifir, where j is a light 
quark jet and £ = e or ji. 



Since a likely 77 collider will have a maximum CM energy of 500 GeV to 1 TeV j25 
the top mass cannot be completely ignored, and we have calculated the full matrix element 
for the 2^2 process, assuming the top quark is on shell. Because various cuts must be 
introduced to simulate a detector and to eliminate backgrounds, we have then calculated 
the top quark decay, using the exact matrix element for the top 3-body semileptonic decay, 
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assuming an on-shell W. Because these couplings do not favor one helicity, the top quark is 
produced unpolarized and we therefore do not have to worry about spin correlations between 
the top production and decay. The cross sections were calculated via a Monte Carlo program, 
using both helicity amplitudes and Dirac matrices, and agreement was found to within 1% 
between the two methods. We have also assumed that BF(t — > bW) ~ 1, which would be the 
case if there were no non-standard top decays other than t — > 57. If there are non-standard 
top decays with a significant branching fraction, the signal results quoted in this paper would 



be reduced at most by a factor of two p3| . For y/s = 500 GeV and m t = 175 GeV we find 
a total signal cross section of 

^(77 -> tq + tq -> bctv) = 76.4 fb ( - ^^v ) ' (4) 

which for a icy (iwy) coupling of k/A = 0.16/TeV (0.72/TeV) translates into 764 (15280) 
events with an integrated luminosity of 10 fb -1 . Therefore we see that this is a viable signal. 

III. ACCEPTANCE CUTS AND BACKGROUNDS 

There are several potentially severe SM backgrounds to the signal. The largest is 77 — > 
W + W~, which has a cross section of about 88 pb at y 7 ? = 500 GeV f2(|. This will mimic the 
signal when one W decays leptonically and the other decays into two light quark jets, giving a 
cross section of about 26 pb. To make a quantitative analysis of the experimental sensitivity 
to the anomalous couplings, we have done a monte carlo study of 77 — > W + W~ — > iuq'q" 
using the full helicity amplitudes |30[| , imposing the following basic acceptance cuts 



f T > 15 GeV, jP T > 15 GeV, EJp iss > 15 GeV, 

\t]% \r) j \ < 1.5, ARij,ARjj > 0.4, (5) 

where px denotes transverse momentum, rj denotes pseudo-rapidity, and AR denotes the 
separation in the azimuthal angle-pseudo rapidity plane. With these basic cuts, the signal 
rate is reduced to about 33 fb, while the W + W~ background is now about 4.8 pb. 
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A significant reduction in the background is possible if 6-tagging is employed. We assume 
a 50% efficiency for detecting a 6-quark jet, and a 0.4% mis-tagging rate where a light 
quark jet is misidentified as a 6-quark [^8[^9j. The signal is then halved, while the W + W~ 
background is reduced a level comparable with the signal. 

Next we can then use the fact that for this background the two jets should reconstruct to 
the W mass, while for the signal the m(bc) invariant mass distribution is peaked towards its 
maximal value of y^ — mw=420 GeV, which can be understood by realizing that the charm 
jet and the b jet from top decay tend to be back-to-back. If we exclude values of m(jj) 
which include the W resonance, we can substantially reduce this background. To quantify 
this in our calculation, we assume a Gaussian energy smearing for the electromagnetic and 
hadronic calorimetry as follows 

AE/E = 30% /VE © 1%, for lepton and photon 

= 80%/^ ©5%, for jets, (6) 

where the © indicates that the ^-dependent and ^-independent errors are to be added in 
quadrature, and E is measured in GeV. If we impose the cut 

m(jj) > 106 GeV, (7) 

then we find that the resonant contributions are reduced to O(l0~ 2 ) fb, effectively eliminat- 
ing this background. The signal is affected very little by the m(jj) cut, as can be seen from 
Fig. 1. 

In addition to resonant W + W~ production, there is a set of Feynman graphs contribut- 
ing to nonresonant 77 — > ivq'q production, where the invariant mass of the lepton and 
quark pairs are not both at the W mass peak. Due to the large number of different graphs 
involved, a precise analytical treatment is beyond the scope of this letter, and indeed has 
not yet been carried out. However, we will make use of the results for resonant and nonres- 
onant contributions analyzed numerically in Ref. JZTJ to show how this background can be 
suppressed. Most of the nonresonant cross section occurs when one pair of fermions is on 
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a W peak and one of the two other fermions is roughly collinear with an incoming photon. 
The pseudo-rapidity cut in Eq. |6], larger than likely required from detector geometry, has 
been chosen to reduce these forward contributions. From Ref. [^7| the cross section for 
77 — > iuq'q" at s/s = 500 GeV, excluding the W + W~ resonant contributions, is about 67 fb 
after the r] cuts (which corresponds to cuts on the charged particles of about | cos^l < 0.92 
in the COM frame of the incoming photons). We then multiplied this result by 8 to account 
for two generations each of leptons and quarks and for positive and negative charged leptons 
in the final state, and divided by 2 since we expect that in roughly half of these events the 
W has on-shell decays into quarks, which will be eliminated by the m(jj) cut. The net 
result is a Wjj — > iujj background of about 270 fb, which will be reduced by more than 
two orders of magnitude by 6-tagging alone. 

A further reduction of the nonresonant background can be made by determining the 
invariant mass m(bW), which for the signal should be strongly peaked near m t and for the 
background should be much flatter. In order to do this reconstruction one must determine 
the W momentum. The neutrino transverse momentum can be inferred from the missing 
Pt, but its longitudinal momentum is undetermined since in general the exact initial photon 
energies may not be known. However, if we assume that the W which decays leptonically 
is on mass shell, one can determine the neutrino longitudinal momentum up to a two-fold 



ambiguity |3T|. We can then take the solution which gives m(bW) closest to m t , where the 



b quark is identified via a 6-tag. Applying the cut 

\m(bW) - m t \ < 30 GeV, (8) 

will then provide a strong constraint on the background, while the signal is generally reduced 
by only about 10-20% BE§]. 



To simulate the effect of the remaining cuts (in p?, AR, m(jj), and m(bW)) on the 
nonresonant background, we have assumed that the distributions that survive the r\ cuts are 
relatively flat. This assumption is reasonable as the region close to the beam pipe, where 



the bulk of the non-resonant cross-section arises 27 , has been excluded. The fraction of the 



non-resonant cross-section which survives is then proportional to the volume of phase space 
allowed after our additional cuts, which is reduced by a factor of 7. If we assume that the 
cross section is correspondingly reduced, we get a nonresonant background of about .16 fb. 
The successive effect of all the cuts on the signal and primary backgrounds are summarized 
in Table 1. 

There are also SM final states bci^v from both resonant and nonresonant backgrounds. 
However, these processes are suppressed by a factor |Vb c | 2 ~ 1/400 compared to the final 
states udt^v and csl v . Although they are reduced only by a factor of 2 when 6-tagging 
is employed, they are still more than a factor of 3 smaller than the corresponding processes 
with light quarks after 6-tagging. We can account for these backgrounds by multiplying the 
light quark background cross section by 21/16, which gives an overall estimated background 
at the 0.2 pb level, which corresponds to 2 events for 10 fb" 1 integrated luminosity. 

The signal cross section after all cuts is about 23.4 fb for k/A = 0.16/TeV. However, 
about half the signal is lost due to the requirement of 6-tagging, still leaving a cross-section 
large enough to be phenomenologically viable, assuming £ = 10 _1 fb. At realistic 77 
colliders, of course, a/s is not 500 GeV as we assumed above, but lower; taking i/i = 
400 GeV, as for example in [2~B|, leads to a reduction of ~ 10% in the signal cross-section 
and should have no significant influence on possible discovery bounds. 



IV. ANOMALOUS COUPLING LIMITS AND DISCUSSION 

To estimate the sensitivity to the anomalous couplings for a given integrated luminosity, 
we require that the signal be observed at the 3-a level, 

S > 3VS + B, (9) 

where S and B are the number of signal and background events, respectively, after all 
cuts are made. In our case (B pa 2), this corresponds to at least 11 signal events. For 
y/s = 500 GeV and an integrated luminosity of 10 fb -1 , the discovery limit for k/K is then 



S 



about 0.048/TeV. For a more realistic CM energy of \/s = 400 GeV, the discovery limit can 
be written approximately as 



when the integrated luminosity C lies in the range 5-20 fb -1 . A 77 collider with maximum 
CM energy of 1 TeV and an effective y/s = 800 GeV may be able to reduce this by about 
10%. We note that if the background has been underestimated by a factor of 3, these 
discovery limits are raised by only about 10%. 

We have also examined the possibility of looking for e + e~ — > tq+iq in an electron-positron 
collider. However, the production cross section is much lower than for a 77 collider. We 
find that for k/A — 0.16/TeV and m t = 175 GeV the total cross section for e + e~ — > tq + iq 
before cuts is about 1.8 fb at y/s = 200 GeV and 4.4 fb at y/s = 500 GeV. These are more 
than an order of magnitude smaller than the corresponding 77 cross sections, and means 
that an e + e~ collider cannot effectively probe the iq'j coupling. 

In summary, we have shown that a 77 collider with y/s = 500 GeV can probe an 
anomalous icy or iwy coupling down to the level of k/A ~ 0.05/TeV for the integrated 
luminosities expected at such machines. As noted previously, a reduction in y/s to 400 GeV 
does not significantly affect this limit. This will be much more sensitive than looking for 
e + e~ — > tc at a similar energy, and is a factor of about 2.5 better than the limit expected to 
be obtained from studying top decays at the upgraded Tevatron, where backgrounds are not 
so easy to suppress. If charm tagging is available it also offers the possibility of distinguishing 
between the anomalous icy and iwy couplings, which plausibly could be quite different. 
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TABLES 

TABLE I. Cross sections in units of fb for the tq,iq — ► bqi^for signal with k/A = 0.16/TeE 
and the SM backgrounds. The successive effect of the cuts on the signal and W + W~ background 
have been explicitly calculated, while the effects on the nonresonant background values have been 
estimated as discussed in the text. The dashes indicate cross sections too small to be of interest. 



(a). Cuts 


signal tc, tc — > bci^v 


WW -► jj£ ± u 


wjj -> jje ± u 


none 


76.4 


26000 


2140 


7] only 


38.2 


6928 


270 


basic+smear 


33.5 


4786 


175 


b-tag 


16.75 


19.0 


0.70 


m(jj) 


15.7 




0.65 


m(bW) 


11.7 




0.16 



FIGURE CAPTIONS 

FIG. 1 The 77 — > tq + tq — > bql^fr cross section versus the jet-jet invariant mass m(jj) 
at y/s = 500 GeV after the basic cuts have been implemented. The solid (dotted) histogram 
corresponds to the result before (after) the effects of detector smearing are included. 
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